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ABSTRACT 


A  DECOMPOSITON  PROCEDURE  FOR  CONVEX  QUADRATIC  PROGRAMS 


C.  M.  Shetty 
Mohammed  Ben  Daya 

School  of  Industrial  and  Systems  Engineering 
Georgia  Institute  of  Technology 
Atlanta,  Georgia  30332-0205 


This  paper  deals  with  the  solution  of  convex  quadratic  programs  by 
iteratively  solving  a  master  problem  and  a  subproblem  as  proposed 
previously  by  Sacher.  The  approach  has  the  advantage  that  the 
subproblems  are  linear  programs  so  that  existing  schemes  for  solving 
large  problems  can  be  taken  advantage  of.  This  paper  gives  a  closed  form 
solution  to  the  master  problem  so  that  the  procedure  is  well  suited  for 
solving  large  quadratic  programs  and  can  take  advantage  of  the  constraint 
structure. 
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1.  INTRODUCTION 


Consider  the  following  quadratic  program 

QP:  minimize  y  xCBx  +  cCx 

subject  to  Ax  >  b 

x  >  0 

where  x  e  R°,  A  e  Rm  n  and  B  is  positive  semi-definite. 

A  decomposition  algorithm  using  Lemke's  method  was  proposed  by 
Sacher  [1]  to  solve  Problem  QP.  The  method  solves  the  original  problem 
by  iteratively  solving  a  master  program  and  a  subprogram. 

In  this  paper,  we  solve  the  master  program  using  the  gradient 
projection  method  of  Rosen  [2]  in  Section  3  and  by  the  Reduced  Gradient 
Algorithm  in  Section  5.  The  methods  take  advantage  of  the  special 
structure  of  the  master  program  and  reduce  the  algorithms  to  simple 
arithmetic  calculations. 

2.  The  Sacher  Algorithm 

Sacher' s  procedure  uses  an  inner  linearization  approach  followed  by 
restriction  and  is  a  nonlinear  version  of  the  Dantzig-Wolfe  decomposition 
principle  [3].  His  algorithm  is  essentially  of  von  Hohenbalken's 
simplicial  decomposition  method  [4]  (5]  when  specialized  to  a  quadratic 
objective  function. 

Algorithm: 

Step  0: 

Let  P  and  Q  be  matrices  whose  columns  are  affinely  independent 
extreme  points  and  directions  respectively  of  the  set  {x:  Ax  >  b,  x  >  0}. 
Here  P  must  have  at  least  one  column,  but  Q  may  be  vacuous. 


(Master  program).  Solve  the  following  problem  by  Lemke's  method. 


MP:  minimize  |  (“)  B  (“)  +  c  (“) 

subject  to  eCu  =1;  u  >  0,  v  >  0 

where  B  =  (P,Q)tB(P,Q)  and  c  =  ( P  C) •  Denote  the  solution  by  [u,v].  If 

Q  C 

the  solution  is  unbounded,  STOP.  The  original  problem  has  an  unbounded 
objective  function. 

Step  2: 

Set  x  =  Pu  +  Qv. 

Step  3: 

(Subprogram).  Solve  the  following  program. 

SP:  minimize  hCx 

subject  to  Ax  >  b,  x  >  0 

where  h  =  BPu  +  BQv  +  c.  If  the  solution  is  bounded  denote  it  by  x. 
Otherwise,  let  x  be  a  (normalized)  direction  of  recession  for  which 
h*’x  <  0. 

Step  4: 

~  t-  t  ~  — 

Stop  if  x  is  an  extreme  point  and  h  x  =  h  x,the  solution  to  QP  is  x 

otherwise  go  to  Step  5. 

Step  5: 

(Updating).  If  u^  =  0  (v^  *  0,  respectively),  delete  column  i  of  P 
(Q,  respectively).  If  x  is  an  extreme  point,  replace  P  by  (P,x). 
Otherwise  replace  Q  by  (Q,x).  Go  to  Step  1. 


3.  Solving  the  Master  Program  Using  the  Gradient  Projection  Method 

We  will  solve  the  master  program  MP  using  the  Gradient  Projection 
Method  (GPM)  of  Rosen  [2].  It  may  be  recalled  that  at  a  feasible  point, 
the  GPM  method  projects  the  gradient  of  the  objective  function  on  the 
null  space  of  the  gradients  of  the  binding  constraints.  It  will  find  an 
improving  feasible  direction,  if  one  exists,  and  a  one  dimensional 
optimization  problem  is  solved  to  find  the  optimum  distance  to  move  in 
the  chosen  direction.  Because  of  the  special  structure  of  the  master 
problem,  closed  form  solutions  to  the  above  can  be  obtained,  and  the 
resultant  algorithm  is  given  below.  Section  4  shows  that  the  steps 
specified  are  precisely  those  obtained  by  the  GPM  method. 

Algorithm: 

Step  0: 

Choose  a  point  (u^.v^)  with  e*^  -  1,  u^  >  0  and  v^  >  0. 

Let  I  =  {i:  u.  >  0}  ;  III  ==  r» 

P  i  P  P 


Id  -  U: 

ui 

>  0}  ; 

I'dl  -  “d 

:  n  =  n  +  n^ 

P  d 

K  -  {i: 

U. 

=  0}  ; 

ic  = 

V'kp  IKP!  ■  p 

P 

1 

p 

* 

ti¬ 

ll 

H* 

Vi 

-  0}  ; 

Kd“ 

!d\Kd  ;  lKdl  ■  d 

Let  k  =  1  and  go  to  step  1. 

Step  1: 

Compute  Vf(u^,v^) 

where  f(u,  v,  )  =  (uk)tB(uk)  +  c(Uk) 

k  k  2  Vv^ 


Let  Vf(u,  ,v, ) 


(vj.v,2 . v") 


Step  2: 


Computer  as  follows; 

0 


dk  =  < 


if  j  e  K  UK. 

p  d 


“k 


-  V 


j 


if  j  e  K  ,  j  =  1,2,. ...n 


if  j  e  K. 


where 


“k  "  *s  V 


jeKp  p 


j 

p'  k 


If  d^  *  0  go  to  step  3. 


If  d^  =  0,  compute  w  as  follows: 


w  =  -  Z  ( — — ) 

0  .  -  '■n  -pJ  k 

jeKp  P 


r  V*  -  E  ( - — )  vj[ 

I  k  . .  -  '■n  -pJ  k 


jeKp  p 


if  ieK 


t  Vk 


if  ieK, 


(1) 


(2) 


(3) 


If  w^  >  0,  the  current  solution  is  optimal  to  MP.  Otherwise,  let 


w  =  min  {w  }  .  Update  K  and  K,  by  deleting  the  index  j,  and  repeat 
J  ieK  UK.  P  d 

P  d 


Step  3: 


Let  A.  =  rain  {A  ,A  } 
k  ’  max 


where 


2(\)t  B  d  +  c  d 
vk  k  K 

A  “  " 


2d 5  B  d,. 


(4) 


and  X  the  maximum  value  of  X  such  that  (u,  ,v  )  +  Xd.  is  nonnegative, 
max  k  k  k 


clearly  X  =  « 
max 

where 


if  d,  >0  and  X  =  min  {XU,XV}  if  d,  ^  0 
k  max  k 


XU  =  min  jeKp,  djj.  <  0} 


XV  =  min  {-v^/d^  :  je^,  d*  <  0} 


If  X^  =  »,  the  original  problem  is  unbounded.  Otherwise,  let 


(»  -  (»  +  M, 


Update  K  ,  K, 

P  d 


Replace  k  by  k+1  and  go  to  Step  2. 

It  may  be  noted  that  at  step  0  of  iteration  2  onwards  of  the  main 

algorithm,  one  could  use  the  optimal  of  the  previous  iteration  as  the 

starting  point.  Also,  in  solving  the  master  problem,  at  each  iteration 

* 

only  K  and  K,  need  be  updated  at  Step  2.  At  Step  3,  in  computing  X  it 
p  d 

may  be  convenient  to  update  B  =  (P,Q)CB(P,Q)  as  follows. 

Consider  the  case  where  the  subprogram  yields  an  extreme  point  p. 

We  must  introduce  a  new  row  and  column  to  B.  Denoting  the  column  of  P(Q) 


by  *s  easily  verified  that  the  new  matrix  is 


l 


5 


a 


i 


if  iel 

P 

if  icld 


e 


'  PtBP1 

.  ptBQi 


if  iel 

P 

if  ield 


9  =  pCBp 

In  the  following  section,  we  give  a  justification  to  the  algorithm 
based  on  the  gradient  projection  method  of  Rosen  and  using  the  structure 
of  the  master  program.  For  more  details  about  the  gradient  projection 
method  see  [6]. 


4.  Justification  of  the  procedure 

Let  M  be  the  matrix  of  the  coefficients  of  the  binding  constraints. 


M  has  the  following  form: 


where  e~  is  a  lxn  vector  of  ones,  D  is  a  pxn  matrix  and  p  =  Ik  I.  Each 
P  p  P 

row  in  D  has  zeros  everywhere  except  a  -1  in  the  jth  column  where  jeK^. 

C  is  a  dxn  matrix  where  d  =  |K,  |.  Each  row  in  C  has  zeros  everywhere 


except  a  -1  in  the  jth  column  where  jeK^. 

It  is  easy  to  show  that  (MtM)  *  is  as  follows: 


where  E  is  an  n  xn  matrix  with  ones  everywhere,  and  I  is  an  ixi 
P  P  1 

identity  matrix. 

Then  it  can  easily  be  verified  that  d^  =  -PVfCu^.v^)  * 

-  [I-Mt(MMt)  ’''M]  Vf(u^,v^)  is  given  by  (1),  and  w  =  (MM*" )  ^MVfCu^.v^)  is 

as  given  by  (2)  and  (3).  Let  iJj(X)  =  f((v^)  +  Xd^J  • 

Solving  - ■  0  for  X  gives  X  of  Eq.  (4). 

5.  A  Reduced  Gradient  Algorithm 

We  now  present  a  reduced  gradient  algorithm  [6]  to  solve  the  master 
problem  MP. 

Algorithm 
Step  0: 

Choose  a  point  y1  *  (Ul)  satisfying 


7 


and  ylk,  d^k  are  the  jth  components  of  y^  and  d^  respectively.  If 
=  <*>,  the  original  problem  is  unbounded.  Otherwise  let  y^+^  -  y^  + 
X^d^,  and  let  k  =  k  +  1,  and  return  to  step  1. 

The  reader  may  note  that  the  procedure  for  updating  B  at  each  main 
iteration  of  the  algorithm  discussed  in  Section  3  holds  in  this  case 
also. 


Summary 

We  have  presented  above  two  algorithms  to  solve  the  master  problem. 
Although  no  proof  is  available  for  the  convergence  of  the  Gradient 
Projection  Method,  it  is  generally  recognized  that  the  method  converges. 
For  the  Reduced  Gradient  Algorithm,  the  reader  may  refer  to  [6]  for  a 
convergence  proof.  Finite  convergence  of  the  main  iteration  of  Sacher's 
algorithm  is  obvious.  At  present  computational  testing  is  on  hand  to 
compare  the  effectiveness  of  the  proposed  procedures.  Other  schemes  for 
special  cases  of  the  problem  are  also  being  developed. 
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